Skip to content

Qualcomm AI Engine Direct - Debugger Convergence Phase 2: Migrating to official numeric discrepancy evaluator#18834

Open
winskuo-quic wants to merge 2 commits intopytorch:mainfrom
CodeLinaro:dev1/winskuo/debugger_convergence_final_stage
Open

Qualcomm AI Engine Direct - Debugger Convergence Phase 2: Migrating to official numeric discrepancy evaluator#18834
winskuo-quic wants to merge 2 commits intopytorch:mainfrom
CodeLinaro:dev1/winskuo/debugger_convergence_final_stage

Conversation

@winskuo-quic
Copy link
Copy Markdown
Collaborator

@winskuo-quic winskuo-quic commented Apr 13, 2026 •
edited
Loading

Summary

Debugger Convergence Stage 2.

Stage 1 (Merged): #17804
Stage 2: This PR
Stage 3: Adding SKILL.md for debugger

Changes made on QNN backend

  • Removed comparator logic and reuse dev tools NumericalComparatorBase
  • Using ETRecrod to retrieve edge_after_transform/forward reference graph
  • Reuse the online edge_after_transform/forward graph instead of the one that goes through serialize and deserialize since serialize will not save quant attributes. Reference:

    executorch/exir/serde/serialize.py

    Line 141 in d31d4be

    def serialize_metadata(self, node: torch.fx.Node) -> Dict[str, str]:
  • Changing UT expected number of events as multi-output node is not supported in dev tools.
  • Verified that the IO's order of the graph is working properly.

Changes made on dev tools

  • executorch/devtools/inspector/_inspector.py

    Line 1150 in 411ede2

    self.op_graph_dict[EDGE_DIALECT_GRAPH_KEY],
    is changed because it is hardcoded to use edge before passes graph for now. Added a param and make sure it is still backward compatible.
  • Added debug_handle to the pandas dataframe since it is helpful for users to map dataframe back to the original graph.

Test plan

Passing E2E test:

  • python backends/qualcomm/tests/test_qnn_delegate.py TestUtilsScript.test_intermediate_debugger --device ${DEVICE} --model SM8750 --build_folder build-android --executorch_root . --artifact_dir ./test_debugger --image_dataset ../datasets/imagenet-mini/val/

Passing the following UT:

  • python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedUtils.test_qnn_backend_dump_intermediate_outputs_simple_model --model SM8750 --device ${DEVICE} --build_folder build-android
  • python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNQuantizedUtils.test_qnn_backend_dump_intermediate_outputs_topk --model SM8750 --device ${DEVICE} --build_folder build-android
  • python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNFloatingPointUtils.test_qnn_backend_dump_intermediate_outputs_topk --model SM8750 --device ${DEVICE}--build_folder build-android
  • python backends/qualcomm/tests/test_qnn_delegate.py -k TestQNNFloatingPointUtils.test_qnn_backend_dump_intermediate_outputs_simple_model --model SM8750 --device ${DEVICE} --build_folder build-android
  • Under devtools/inspector/_inspector_utils.py, skip delegate call event since it holds all debug_handles and will mess up the op event debug handle.

@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Apr 13, 2026 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18834

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures, 2 Unrelated Failures

As of commit 5ecde09 with merge base 7fdd306 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 13, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 13, 2026

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@winskuo-quic
Copy link
Copy Markdown
Collaborator Author

winskuo-quic commented Apr 13, 2026 •
edited
Loading

Hi @Gasoonjia,
This PR is the 2nd stage of the debugger convergence. I have made some minor changes on the dev tools side and have listed the files and reasons for the change. Please have a look.
Thanks

@winskuo-quic winskuo-quic force-pushed the dev1/winskuo/debugger_convergence_final_stage branch from 00b62ad to a3760f4 Compare April 14, 2026 02:09
@winskuo-quic winskuo-quic force-pushed the dev1/winskuo/debugger_convergence_final_stage branch from a3760f4 to 5ecde09 Compare April 20, 2026 01:57
@winskuo-quic winskuo-quic changed the title Qualcomm AI Engine Direct - Debugger Convergence Qualcomm AI Engine Direct - Debugger Convergence Phase 2: Migrating to official numeric discrepancy evaluator Apr 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@abhinaykukkadapu abhinaykukkadapu Awaiting requested review from abhinaykukkadapu abhinaykukkadapu is a code owner

@Gasoonjia Gasoonjia Awaiting requested review from Gasoonjia Gasoonjia is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@winskuo-quic